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Film Grain Simulation and Comfort Noise Addition 
Specification for DirecTV A3 System 



1 INTRODUCTION 

This document provides a specification for film grain simulation and comfort noise addition in a 
DirecTV A3 system. 

The film grain simulation is based on the contribution JVT-I013r2 [1] adopted at the 7th JVT 
meeting. Constraints of the DirecTV system regarding the value of the parameters of the SEI 
message and some implementation aspects are described. 

Comfort noise addition reuses hardware elements of the film grain simulation to hide 
compression artifacts. 



2 FILM GRAIN SIMULATION 

2.1 FILM GRAIN SEI Message Constraints 

The film grain generator makes use of the parameters specified in the SEI message described in 
[1, 2]. Limitations regarding the SEI message are described. 

Film grain SEI messages may only be sent preceding I pictures, and only one film grain SEI 
message may precede a particular I picture. I pictures are indicated by slice_type equal to 7, or 
by nal_ref_idc equal to 5. 

model_id shall be 0. This identifies the film grain simulation model as frequency filtering. 

colour_space_id shall be 0. This identifies the color space in which the parameters of the 
SEI message have been estimated as YCbCr. 



i 

i 

blending_mode_id shall be 0. This identifies the blending mode used to blend the 
simulated film grain with the decoded images as additive. 

log2_scale_factor shall be in the range [2, 7]. 

compl_param_present_f lag shall be 0. This prevents the transmission of film grain 
parameters for the Cb color component. 

comp2_param_j>resent_flag shall be 0. This prevents the transmission of film grain 
parameters for the Cr color component. 

no_int ens i ty^in t erval s^minus 1 [0] shall be in the range [0, 7]. This gives the 
number of intensity intervals for which a specific set of parameters has been estimated. 

intensity_interval_lower_bound [ 0 ] [i + 1] > 

intensity_interval_upper_bound [ 0 ] [ i] for i=0. . .6. This indicates that multi- 
generational film grain is not supported. 

no_param_minusl [0] shall be in the range [0,2]. Low-pass modeling and cross-color 
correlation are not used. 

param[0] [±]' [0] shall be in the range [0, 255]. 
param[0] [i] [1] shall be in the range [3, 15]. 

param[0] [i] [2] shall be in the range [3, 15] and shall be transmitted only when not equal to 
param[ 0 ][ i ][ 1 ]. When both parameters are transmitted, the allowed only the pair of values 
ere-listed in Table 1 are allowed . 
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15 ; 



Table 1. 



All the other parameters of the SEI message have no constraint with respect to the standard 
specification. 

2.2 Film grain Implementation aspects 

Film grain addition involves two distinct steps of operation. First, an initialization process is 
performed when an SEI message is received preceding an I picture, in which a pool of film grain 
blocks is created. Then, prior to display, a simple process is applied to add the stored film grain 
blocks to each luma pixel of each decoded picture. 

2.2.1 Initialization at SEI message receipt to create film grain block pool 

Upon receipt of a film grain SEI message, an initialization process is performed to create a pool 
of 4,096 (512x8) film grain pixel values for each of up to 8 different luma intensity intervals. 
The number of luma intensity intervals is indicated by 1 plus the SEI message field 
no_intensity_intervals_minusl [0] . Generation of the film grain samples begins 
with the lowest luma intensity interval. 

Bit-accurate simulation of the film grain noise can be accomplished using a specified uniform 
pseudo-random number generator polynomial and using a specified database of film grain 
patterns. The database of film grain patterns is composed of 13 sets of 4,096 (512x8) values 
each. The values are stored in 2's complement form and range from [-127, 1271. The list of 
values for each set is shown in the Appendix. 

The process to obtain the film grain pixel values for a particular luma intensity interval s is 
shown in the block diagram of Figure 1 . The process specifies the access to the database of film 
grain patterns, the scaling of the values, and their storage into the pool as follows: 

for(i = 0..4,095) 

v = param[0][s][0] * database[ m ][ n ][ i ] 

pool[ s ][ i ] = (((v + 2 los2 - scale - faclor - ') » 3og2_scale_f actor ) + 32) » 6 
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where n is equal to param[ 0 ][ s ][ 2 ] - 3, m is equal to 0 when no par am minusl [01 is 1 
and equal to 1 otherwise, and the factor 6 scales the film grain values provided in the Appendix. 

This process is performed as many times as indicated by 1 plus the SEI message field 
no_intensity_intervals_minusl [0] . 

2.2.2 Block and pixel operations prior to pixel display 

The operations performed to add film grain to the decoded picture at block and pixel level are 
shown in Figure 2. For each 8x8 block of the decoded image, the average of the luma pixel 
values is computed and compared to the SEI message intensity__interval_low- 
r_bound[0] [i] and intensity_interval_upper_bound[0] [i] parameters to 

determine the correct luma intensity interval for the block. 

A uniform random number generator, using a primitive polynomial modulo 2 operator, x J8 + x 5 + 
x 2 + x 1 + 7, is used to select film grain blocks from the pool. Let x(i, e) indicate the i-th symbol 
of the sequence x, beginning with an initial seed e. (The seed is set to 1 upon the receipt of each 
film grain SEI message.) The offset for the current 8x8 block of film grain is generated as 
follows: 

previous_offset = offset 
offset = (x(i, 1) % 4 ; 088 )» 2 
offset A = (index == previous_offset) 
offset «= 2 

where offset has been initialized to 0 after the creation of the pool. After the calculation of the 
offset, the 8x8 block of film grain is extracted from the pool as follows: 

for(i_=0..7,j = 0..7) 

block[i][j] = pool [s] [offset + i + j*4096] 

Before blending the film grain block with the decoded image, deblocking of the pixels on the left 
and right columns of the block is performed as described in Section 2.2.2.1. The deblocked film 
grain block is then added to the decoded pixels and the result clipped to [0, 255] for display. 
Film grain noise is only added to luma pixels. 




2.2.2.1 Deblocking filter 

As suggested in [I], a deblocking filter shall be applied on the film grain image before blending 
to smooth the blocking artifacts resulting from the small size of the transform. The deblocking 
filter is implemented by means of a 3-tap filter applied to all pixels bordering the 8x8 block left 
and right edges. Given a row of pixels belonging to two adjacent 8x8 blocks, the transition 
between blocks being located between pixels b and c, 



I 



Block A 



Block B 



the filter is applied as follows: 

b'=(a + (b « l) + c)»2 
c'=(b + (c « l) + d) »2 



where b' and c' replace the value of the original pixels b and c, respectively. Deblocking of the 
left and right block edges is done for every block at display time. 



3 COMFORT NOISE SPECIFICATION 

Comfort noise addition is used to hide compression artifacts. Comfort noise addition and film 
grain simulation are not used at the same time. Unlike film grain simulation, it is not intended to 
match a pre-specified noise pattern (i.e. film grain). 

A custom SEI message is proposed to enable turning comfort noise on and off, as well as 
indicating the level of noise to add, based on the expected level of compression artifacts. 

Comfort noise addition utilizes several of the film grain simulation hardware elements. 
3.1 Comfort noise SEI message 

We propose use of a registered user data SEI message to indicate the use of comfort noise. It 
applies to all pictures that follow, until an IDR or a new comfort noise or film grain SEI message 
arrives. Comfort noise SEI messages may only be sent preceding I pictures, and only one 
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comfort noise SEI message may precede a particular I picture. I pictures are indicated by 
slice_type equal to 7, or by nal_ref_idc equal to 5. 



user_data__registered_itu_t_t35( payloadSize ) { 


C 


Descriptor 


itu_t_t35_country_code 


5 


b(8) 


itu_t_t35_payload_byte 


5 


b(8) 


comfort_noise_flag 


5 


u(1) 


if (comfort_noise_flag == 1) { 






comfort_noise_qp_offset_idc 


5 


ue(v) 


comfort_noise_qp_vveight_offseMdc 


5 


ue(v) 


} 






) 







comfort_noise_flag equal to 1 indicates that comfort noise addition is used. 
comfort_noise_flag equal to 0 indicates that comfort noise addition is not used. 

comfort_noise_qp_offset_idc indicates the quantization parameter offset used in the calculation 
of the additive comfort noise level, and may range in value from -51 to 52. 

comfort_noise_qp_\veight_offset_idc indicates a quantization parameter weight offset used in 
the calculation of the additive comfort noise level, and may range in value from -6 to 7. 

3.2 Comfort noise implementation aspects 

Comfort noise addition includes operations performed at the block level, and operations 
performed at the pixel level. A key difference between film grain and comfort noise is the 
temporal correlation of additive comfort noise. 

3.2.1 Block level operations 

Per block operations are performed to calculate relative weights of the three terms used for 
comfort noise generation, as shown in Figure 3. The inputs to this process are decoded luma 
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pixels, the picture QP = (pic_init_qp_minus26 + 26), and the comfort_noise_qp_offset_idc 
and comfort_noise_qp_weight_offset_idc from the comfort noise SEI message. 

The current picture number is indicated by t. t is reset to 0 for the I picture that follows a comfort 
noise SEI message. The average of the 8x8 luma pixel block of the current picture, t, is 
calculated as block_avg(t) and compared to a threshold. If block_avg(t) ^<_10, block_avgJevel [ 
= 1, otherwise block_avg_level = 0. 

The absolute difference of the current luma block average with respect to the co-located block 
from the previous displayed picture is calculated, where t-1 indicates the previous displayed 
picture, and compared to a threshold. If lblock_avg(t) - block_avg(t-l)l > 3, block_absdiff Jevel 
= 0, otherwise block_absdiff_level = 1. If t is equal to 0, block_absdiff_level = 1. 

For SD resolutions and below, all pixel and block operations are done using the display 
resolution. For HD resolutions, block operations are performed using 2x2 sub-sampled pixels 
(using the upper left pixel of 2x2 pixels), so the 8x8 luma pixel avg involves adding 8x8 = 64 
pixel values, but these values are spread over a 16x16 pixel range. For SD resolutions, storage 
of the block_avg values for the entire picture requires storage of 1/64 the size of a frame store. 
For HD resolutions, storage of block_avg values requires a storage of 1/256 the size of a frame 
store. 

The values of comfort noise qp offset idc. comfort noise qp weight offset idc, and picture 
OP, are used in the calculation of an intermediate weight w 3 which is calculated as follows: 

w n = (c1ip((weight(clip(OP +comfort noise qp offset idc,0,5iy) + 
comfort noise qp weight offset idc),0/7) 



where weightfO) is defined in Table 2 Table K 
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Table 24. weightfOY lookup table 
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The value of w Q . which needs to be computed only once for each picture, and the values of 
block avg level and block absdiff level are then input to a lookup table to find the values of the 
final weights, w n, w_i , and w _ f . which are used in the pixel level operations. 
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Table 3^. Lookup table for comfort noise generation. 
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f>U6*>>6? > b% 



Note that the determination of the current picture structure is based on display rather than coding 
structure. For interlaced display, pixels from the current field and previous field are used in the 
calculations. For progressive display, pixels from the current frame and previous frame are used 
in the calculations. 

The values of block_avgJcvol and block_absdiff Jcvcl arc input to a lookup tabl e to find the 
values of swqt^w^ and sw p 
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Table 2. Lookup table 
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Tho s e values [ sw otSw^tSw^ ] and the values of th e picture QF, comfort_noisc_qp_off s ct_ide, 
and comfort_noisc_qp_weight_offsct_idc are then used in the calculation of the final weights, 
WQ7-W4- and w ^ ? which are used in the pixel level operations. 
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Table 3. vvcight(Q) lookup table 



With wcight(Q) defined in Table 2, 

Wq = (clip((wcight - (clip(QP i comfort_noi s e„qp_off s ct_idc,0,5 1)) i 
conifort_noisc_qp_wcight_offsct_idc),0,7) 



Wq = SWq-^-W^ 
W4. a SW 4^-W q 

The abo\'e multiplies could b e coinpletely avoided if implemented with the use of a larger lookup 
table that also considers the value of w q , and which can immediately compute Wg^w^., and w $ 
without the intermediate calculations of sw qtSw^ , and sw p 
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Tabic 4 . Expanded lookup tabic for multiplication avoidance. 

Note that the determination of the current picture structure is based on display rather than coding 
structure. For interlaced display, pixels from the current field and pr e vious field are u s ed in the 
calculations. For progressive display, pixels from th e current frame and previous frame are used 
in the calculations. 

3-2.2 Pixel level operations 

The comfort noise pixel level operations use the same pre-stored Gaussian random number list 
and primitive polynomial generator as the film grain simulation. Three distinct primitive 
polynomial uniform random number patterns are used within the comfort noise generation 
process. 

For SD resolutions, the pixel level operations are performed for all luma pixels, and the 
generated luma noise value is applied to both luma and chroma pixels. Let pic_width = 
PicWidthInSamples L . 

For HD resolutions, the pixel level operations are performed on Va of the pixels, using a 2x2 sub- 
sampling, and then the generated noise value applied to all pixels, using a 2x2 pel repeat. Let 
picjwidth = PicWidthInSamples L » 1- 



lb - BEST AVAILABLE COPY 



The first uniform random number pattern x 0 (i, seed_x 0 (t)) is initialized with a seed, 
seed_x 0 (0)=3, at the arrival of the comfort noise SEI message. At each new frame t the uniform 
random number pattern is initialized to seed_xo(t) where seed_Xo(t)= seed_Xo(t-l) + 2. 

The second uniform random number pattern X|(i, seed_X](t)) is initialized with seed_Xj(t)= 
seed_x 0 (t-l). This implies that Xi(i, seed_X](t))= xo(i, seed_xo(t-l)). 

Two new numbers UN 0 (t,i) and UNj(t,i)= UN 0 (t-l,i) are then generated from these two patterns 
as follows: 

UN 0 (t,i) = x 0 (i, seed_x 0 (t)) % 32 - 16 

UNi(t 5 i) = UN 0 (t-l,i) = xi(i, seed„x,(t)) %32- 16 

The third uniform random number generator, UN f , is initialized to 1 at the beginning of each 
displayed frame and is used to generate a fixed noise image. This number generates offsets into 
the Gaussian J ist[ ] to access a line of 8 random numbers using the following operations, where i 
increments for each 8 values: 

UNf=x(i,l) 

for n=CL7, G[n] = (Gaussian_list[(UN f + n)%2048] + 1) » 1 
The noise value at position [r][s] of the noise image is computed as 
m = pic_width*r+s 

noise[r][s] = (w f *G[s%8]+ w 0 *UNo(t, m) + w,*UN 0 (M, m) + 512 ) » 10 
where w 0; w i , and w f change at the block boundaries. 

For SD sequences the final noise luma_noise is identical to noise, while for HD sequences the 
final noise luma_noise is generated by performing a 2x2 upsampling of noise using pixel 
repetition. 

The chroma noise is half the value of the final luma noise 

chroma_noise[r][s] = (luma_noise [r*2][s*2] + 1 ) » 1 

Finally comfort noise shall be added to the decoded pixels and the result shall be clipped within 
the range of [0, 255] for display. 
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Figure 1. Film grain initialization process at SEI message receipt, 
performed for multiple blocks in each luma intensity interval 
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Figure 2. Film grain per block and per pixel operations 



\5~ ' 



Decoded 
fuma pixels 
► 



Luma 8x8 block 
avg 



block_avg(t) 



Delay 



block_sum(t-1) 



Threshold test 



block_sum_ 
level 



Absolute 
difference 



Threshold test 



block_absdrrf_ 
level 



QP 



oflset 



weight_ortset 



Lookup table 



Lookup table 



Figure 3. Comfort noise per block operations 
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Figure 4. Comfort noise per pixel operations 
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